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Abstract 

Achieving joint objectives by teams of cooperative planning agents 
^3 requires significant coordination and communication efforts. For a single- 

O agent system facing a plan failure in a dynamic environment, arguably, 

attempts to repair the failed plan in general do not straightforwardly bring 
I any benefit in terms of time complexity. However, in multi-agent settings 

the communication complexity might be of a much higher importance, 
possibly a high communication overhead might be even prohibitive in 
£■ — certain domains. We hypothesize that in decentralized systems, where 

f" — coordination is enforced to achieve joint objectives, attempts to repair 

CNI failed multi-agent plans should lead to lower communication overhead than 

replanning from scratch. 

The contribution of the presented paper is threefold. Firstly, we for- 
mally introduce the multi-agent plan repair problem and formally present 
i-H the core hypothesis underlying our work. Secondly, we propose three al- 

L* gorithms for multi-agent plan repair reducing the problem to specialized 

. ^ instances of the multi-agent planning problem. Finally, we present results 

of experimental validation confirming the core hypothesis of the paper. 

1 Motivation 



Classical planning and multi-agent planning based on classical planning are ap- 
proaches to constructing autonomous agents and teams of agents, which attempt 
to achieve their objectives in an environment. The result of the planning process 
is traditionally a plan, a sequence of actions the agent should perform in order 
to achieve a given goal. When the agent is situated in a dynamic environment, 

"This is the full version of an extended abstract published in Proceedings of the 11th 
International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), 
Conitzer, Winikoff, Padgham, and van der Hoek (eds.), June, 4-8, 2012, Valencia, Spain. 
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occurrence of various unexpected events the environment generates might lead 
to the plan invalidation, a failure. A straightforward solution to this problem is 
to invoke a planning algorithm and compute a new plan from the state the agent 
found itself in after the failure to a state conforming with its original objective. 

Planning, as well as replanning, in the case of a failure occurrence, is a 
costly procedure, especially in terms of its time complexity. It is relatively 
straightforward to see, that in many cases, however, a relatively minor fix to 
the original plan would resolve the failure possibly at a lower cost. Because 
it is not clear what exactly are the planning domains and types of dynamic 
environments which would allow for such a repair approach, it can be argued 
that non-informed plan repair attempts can in many cases even raise the overall 
complexity of the approach in comparison to replanning. This would be due 
to futile attempts to repair the failed plan before inevitably falling back to 
replanning. 

In general, plan repair can be seen as planning with re-use of fragments of the 
old plan. There is a number of works, empirically demonstrating that plan repair 
in various domains performs better than replanning (e.g., |S1|51|S]). However, 
in [nj|, Nebel and Koehler theoretically analyzed plan re-use (plan repair), and 
conclude that in general it does not bring any benefit over replanning in terms 
of computational time complexity. 

In situated multi-agent systems, however, the time complexity is often not 
of the primary importance. In such systems, often it is the communication com- 
plexity which is a higher priority concern. Consider application domains, such 
as e.g., undersea operations by teams of coordinated autonomous underwater 
vehicles. While the state-of-the-art technology allows to employ relatively pow- 
erful computers on board of such robots, the communication links are extremely 
constrained and expensive; wireless networks cannot be deployed and commu- 
nication is performed mostly using acoustic signaling. In such applications, it 
is the communication complexity of the distributed planning algorithms which 
matters more than time complexity. Consequently, employment of multi-agent 
plain repair techniques can provide a tangible benefit over replanning for a team 
of robots whose multi-agent plan fails. 

The motivation for our research is the intuition that multi-agent plan repair, 
even though not always the fastest approach, should under specific conditions 
generate lower communication overheads in comparison to replanning. The 
conditions correspond to the level of required coordination and the types of 
failures the environment generates. While the hypothesis is rather intuitive, 
our approach is significant in that we give it a rigorous treatment. Besides 
our preliminary approach in [7], this aspect of multi-agent planning and plan 
repair, while obviously important with arrival of truly distributed algorithms 
for multi-agent planning, such as the one by Nissim et al. did not witness 
considerable attention of the community yet. 

The contribution of the the presented paper is threefold. Firstly, after intro- 
ducing the general problem of multi-agent planning stemming from the formu- 
lation due to Brafman and Domshlak [3] in Section [2] in subsequent Section [3] 
we formally introduce the multi-agent plan repair problem and formally state 
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the core hypothesis of the presented research. Secondly, still in Section [3] we 
propose three algorithms for multi-agent plan repair reducing the problem to 
specialized instances of the multi-agent planning problem. Finally, in Section [4] 
we present experimental validation confirming the core hypothesis of the paper. 
Section [5] concludes the paper by some final remarks regarding the shortcomings 
of our approach and future outlooks in the here described line of research. 

2 Multi-agent planning 

Classical, single-agent planning problem is characterized by a set of states with 
a unique initial state, a final state (a set of final states) and a set of actions 
representing the transitions between these states that the system undertakes 
upon performing the actions. We define multi-agent planning problem as an 
extension of the classical single-agent planning. We consider a number of co- 
operative and coordinated actors featuring possibly distinct sets of capabilities 
(actions), which concurrently plan and subsequently execute their local plans 
so that they achieve a joint goal. 

An instance of a multi-agent planning problem is defined by: i) an environ- 
ment characterized by a state space, ii) a finite set of agents, each characterized 
by a set of primitive actions (or capabilities) it can execute in the environment, 
iii) an initial state the agents start their activities in and iv) a characterization 
of the desired goal states. Before treating the problem of multi-agent planning 
and summarizing a state-of-the-art algorithm for solving it, we first formally 
introduce the underlying concepts. 

2.1 Preliminaries 

Consider a set of atoms {pi, . . . ,p n }. A state is a set of terms from a language 
C = {pi, . . . ,p„, ->pi, . . . , ^p„} where ^p denotes a negation of p. We also 
assume the standard tautology p = ^^p for every p G C. Furthermore, we 
require all states to be consistent, i.e., for a state s C C we have that p G s if 
and only if ^p ^ s. Note, in general, the states do not have to be complete, i.e., 
it might be that there is a p G C, such that {p, ^p} (£ s. S denotes the set of all 
states and we assume there is a distinguished state ^65 denoting an undefined 
state in which the overall system can be in. To simplify the notation, we also 
extend the negation to states as follows — {p|^p G s}. The set of atoms 
corresponding to a set of terms C £ is denoted <j> = {p | p € <j> or ->p € </)}■ 

A primitive action, or simply an action, is a tuple <f>atf>, where a is a unique 
action label and <f>,ip C C respectively denote the sets of preconditions and 
effects of a. The preconditions and effects are assumed to be consistent sets of 
terms. Whenever the context is clear, we simply write a instead of (paip. Act 
denotes the set of all actions and we furthermore assume there is a distinguished 
empty action 0e0 G Act with no preconditions and no effects. 

We say that an action ifiaip is applicable in a state s iff (ft C s. An application 
of (j)atp is defined by the state transformation operator : S x Act — > S defined 
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as follows: 

{(sUip)\ iff <t> is applicable in s, 

X otherwise. 

Note that given a consistent state s and an action ifiaip, the application of a 
to s results in either the undefined state \i or > m the case a was applicable in s, 
a consistent state s' again. The application of <j>aip to s first enriches s with all 
the effects of a, however, in the case there exists some p£s, s.t., ^p e the 
simple unification would make the resulting state inconsistent. The subsequent 
set subtraction of terms which were the source of such inconsistencies makes the 
resulting state consistent again, while at the same time preserving the effects of 
a. Furthermore, © is associative, hence we can write s © a\ © • • • © a n . 

An agent a = {(f> ai aiip ai , ■ ■ ■ , 4 l a n a,n^Pa n } is characterized precisely by its 
capabilities, a finite repertoire of actions it can preform in the environment. 
From now on, we assume that there exists a language C giving rise to a state 
space S. 

2.2 The Problem of Multi-agent Planning 

Definition 1 (multi-agent planning). A multi-agent planning problem is a tuple 
II = (A, ,sq, S g ), where A is a set of agents a\ , . . . , a n , featuring mutually disjoint 
sets of actions, an initial state so € S and a set of goal states S g C S. 

Before formally defining the notion of a solution to a multi-agent planning 
problem, we first introduce a sequel of auxiliary notions. 

Given an agent a, a single-agent plan P is a sequence of actions a\, . . . , a^, 
s.t., fljEa for every i. P[i] denotes the i-th action in P, or P[i] = e in the case 
i is larger than the length of P, which in turn will be denoted \P\. 

A team of agents A = a±, . . . , a n can act in the environment concurrently 
A joint action ^ai/'a of the team is specified by a = (4> ai ai'ipa 1 , ■ ■ ■ , 4>a n anipa n ) 
a tuple of actions corresponding to the individual agents, i.e., <Xj G a,- L for each 
i, its preconditions a = U i=1 4>at an d its effects "0a = Ui^iV'ai- a [^] denotes 
the fc-th action of a. Similarly to actions of individual agents, a and -0a are 
assumed to be consistent sets of terms. The notion of action applicability in 
a state s, as well as application of a to s straightforwardly extend from the 
definitions for primitive actions. 

Definition 2 (multi-agent plan). Let II = (A,s a ,S g ) be a multi-agent plan- 
ning problem with A = ai,...,a n . A synchronous multi-agent plan V = 
{Pi, . . . , Pn}, consisting of single agent plans Pi, . . . , P„ respectively constructed 
from actions of the agents «i, . . . , a n is a solution to II if the plan V satisfies 
the following: 

1. V is well-formed, i.e., \Pi\ = \Pj\ for all i, j < n. \V\ = \Pk\, for some 
k < n, denotes the length of the multi-agent plan V; 

2. V is feasible, i.e., there exists a sequel of states si, . . . ,s m , s.t. m = \P\ 
and Sj + i = Si © a^ with a^ = (Pi[i], ... ,P n [i\) for all i < m; and finally 
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3. V reaches the goal S g , i.e., there exists s g € S g , s.t. s m C s g . 

We also say that V solves the problem II. Finally, Plans(U) denotes the set of 
plans which arc solutions to a given multi-agent planning problem II. 

Additionally, V[k] denotes the joint action of the team in the step k and 
V[k, i] denotes the primitive action of the agent i in the step k. This notation 
allows us to introduce the following plan-matrix notation for a multi-agent plan 
V, which provides a more visual understanding of multi-agent plans: 



V 



f an 0-21 ' ' ' 0"ml \ 
ai2 d22 0-m2 

\ O-ln 0,2n ' ' ' 0, mn J 



where — V[i,j]. Indices i (i < m = \V\) and j (j < n) denote the step of 
the plan V and the agent which performs the primitive action, respectively. 

We say that two multi-agent plans Pi, V2 are equal (Vi = V2) iff they have 
the same length (\Vi\ = IT^I) and for all i and j we have Vi[i,j] = ^[hj]- 

A concatenation of two multi-agent plans V\ and V2 over the same agents 
ai,...,a n is defined as a plan V = Vi ■ V2, where for each i and j we have 
V[i,j] = Vi[i,j] if i < \Vi\ and V[i,j] = V 2 [i ~ \V 2 \J] for i > \Ti\. In the 
plan-matrix notation, the concatenation would correspond to simple columns- 
appending operation. Note, concatenation of multi-agent plans is an associative 
operation. 

Given a multi-agent plan V , V[i..]\ denotes a fragment of V from the step i 
to the step j. More precisely, V\i..f\ is a fragment of V iff there exist multi-agent 
plans V pre fi x and V su ffi x , such that V pre fix • V[i..j] ■ V suffix = V . Finally, V[i..oo] 
denotes the i-th suffix of the plan V, i.e., V[i..oo] = V[i..\V\]. Vi ■ V2 is said to 
be a decomposition of a multi-agent plan V iff V = V\ ■ V2 ■ 

Given two multi-agent plans Vi and V2 we can define how different they 
are. diff(Vi,p2) denotes the difference between Vi and V2, that is the overall 
number of primitive actions in Vi , which do not correlate with the correspond- 
ing primitive actions in V2 and vice versa. Formally, diff(Vi,V2) = | 
Vi[i,j] + P 2 [i,j}}\- In the case \Vi\ < \V 2 \, diff{Vi,V 2 ) = diff{Vi ■ P e ,P 2 ), 
where V e is a plan padding of V\ to the overall length \Pi -V £ \ — IP2I an d filled 
with empty actions, i.e, for each we have T e [i,j] — e. Note that the measure 
diff is position agnostic, i.e., we define diff (Vi, V2) = diff(V2,'Pi) in the case 

\Pi\ > \r 2 \. 



2.3 Planning Algorithm 

The above formulation of the multi-agent planning problem is well in line with 
the original formulation of MA-Strips planning due to Brafman and Domsh- 
lak [3]. The authors there additionally distinguish between the public and pri- 
vate actions of the individual agents. An action is public whenever its precondi- 
tions or effects involve atoms occurring in preconditions or effects of an action 
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belonging to another agent of the team. Formally, given a multi-agent team 
A = a\, . . . , a n , the set of public actions is defined as Act pub — {a \ 3i, j : i ^ 
j, a € on, a' 6 a?, and (<f> a U ijj a ) f] (4> a > Ui/v) 7^ 0}- Recall, <j) denotes the set of 
non-negated atoms occurring in (j). Act pnv = Act \ Act pub , where Act = 1J" =1 on 
is the set of all actions the team A can perform. 

The distinction of actions to private and public turns out to be an impor- 
tant one. Since private actions do not depend, nor are dependencies of other 
actions performable by the team, planning of sequences of private actions can 
be implemented strictly locally by the agent the actions belongs to. In effect, 
the public actions become points of coordination among the multi-agent team 
members and a truly decentralized multi-agent planning algorithm for a plan- 
ning problem II can be implemented in two interleaving stages until a suitable 
multi-agent plan is found: i) a plan consisting exclusively of public actions of the 
agent team is calculated, and subsequently ii) the sequences of private actions 
between the public actions of each individual agent are computed to fill in the 
gaps. 

The main contribution of the Brafman and Domshlak's paper lies in pointing 
out that the algorithms can be implemented by reduction of the first stage of 
the planning process to a constraint satisfaction problem (CSP) corresponding 
to the multi-agent planning problem with public actions only. The second stage 
can be subsequently solved by any classical single-agent planning algorithm. In 
result, solving a given multi-agent planning problem can be loosely formulated 
as a CSP with the following two types of constraints: 

coordination constraint: a sequence of joint actions V (candidate multi- 
agent plan) satisfies the coordination constraint iff for every action (j) a aip a — 
V[k,i] performed by the agent on in the step k we have, that if a is a public 
action, then 

• for every p £ <fi ai there must exist ^>a v o, p i\)a v = V[k p ,ip\, such that 
P G V'a,, and < k p < k (there is some previous action which causes 
p to hold), and 

• for no k', s.t., k p < k! < k there exists ip'a'ip' = V[k',i'}, such that 
~^p G tp' (p won't be invalidated between causing it in the step k p and 
execution of a in the step k) . 

internal planning constraint: a sequence of joint actions V satisfies the in- 
ternal planning constraint iff for every agent, the corresponding single- 
agent planning problem with landmarks {a \ a — V[k,i] £ Act pub } is 
solvable. I.e., a single-agent planning algorithm is able to fill in the gaps 
between the public actions in the candidate multi-agent plan. 

Algorithm [T] lists the original multi-agent planning algorithm MA- Plan by 
Brafman and Domshlak in [3] . The algorithm iterates through CSP formulations 
of the planning problem according to S, informally the number of coordination 
points between the agents in the multi-agent team. I.e., S determines the number 
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Algorithm 1 MA-Plan(II): 



Input: A multi-agent planning problem II = (.4, so, S g ). 
Output: A multi-agent plan V solving II, if such exists. 

<5=1 
loop 

Construct CSPrL.4 

if solve-csp(CSPn ; <5) then 

Reconstruct a plan V from a solution for CSPn : <5- 
return V 
else 

5=5+1 
end if 
end loop 



of joint actions in a candidate multi-agent plan consisting of only public actions. 
Filling the gaps between the individual single-agent public actions, if possible, 
then gives rise to the overall multi-agent plan. In the case such a plan completion 
does not exist, the process continues by testing longer candidate plans. 

The original multi-agent planning algorithm assumes a centralized planning 
architecture. I.e., it is a centralized planning algorithm computing multi-agent 
plans for a team of agents which are supposed to be subsequently executed in a 
decentralized fashion. Our motivation is however a decentralized planning/plan 
repair algorithm followed by a decentralized plan execution. 

Nissim et al. in [11] adapted the original blueprint algorithm described 
above to a distributed setting. The adaptation rests on formulating the multi- 
agent planning problem as a distributed constraint satisfaction problem instance 
(DisCSP) and subsequently utilizing a a state-of-the-art DisCSP solver for solving 
it, plus managing the overhead involved in the resulting distributed algorithm. 
The resulting algorithm, however, closely follows the scheme of the original 
algorithm as listed in Algorithm [T] From now on, whenever we speak about 
the implementation of the multi-agent planning algorithm, we have in mind its 
decentralized version due to Nissim et al. [IT] . 

3 Multi-agent plan repair 

Consider a multi-agent planning problem II = (A, sq, S g ) and a plan V solving 
II. Furthermore, consider an environment in which, apart from the actions 
performed by the agents of the team A, no other exogenous events occur. We 
say that such an environment is ideal, or non-dynamic. The execution of V in 
such an environment is uniquely determined by the set of states sq, . . . , s m , such 
that Si+i = Si ®V[i] (cf. also Definition |2| . 

In dynamic environments, however, it can occur that in the course of exe- 
cution of V, the environment interferes and the execution of some action V[i] 
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from the plan V does not result in precisely the state Si+i as defined above. 
We could say that at step i an unexpected event occurred in the environment. 
For simplicity, we consider only unexpected events happening exclusively in the 
course of execution of some action (as if it took a non-zero time), not such 
which could occur while the agent is deliberating the execution (i.e., as if the 
deliberation was instantaneous). 

Note that not all unexpected events in a dynamic environment necessarily 
lead to problems with execution of the plan V. However, there are at least two 
cases of such events, which can be considered a plan execution failure. 

A weak failure of execution of the plan V at step i w.r.t. the multi-agent plan- 
ning problem II is such, when the state s/ resulting from an attempt to perform 
an action (j) a aip a = V [i] for some i docs not satisfy some of the postconditions 
of a, i.e., Va % s f . 

A strong failure of execution of the plan V at step i w.r.t. the planning 
problem II occurs whenever the i-th action of V cannot be executed due to 
its inapplicability. I.e., the execution of the plan up to the step i resulted in 
states so, s[ . . . , s' i: possibly with some weak failures occurring in the course of 
execution of the plan fragment and V[i] is not applicable in s^. 

The weak and the strong plan execution failures are, however, just two ex- 
amples of a plan failure. There certainly arc application domains in which weak 
failures can be tolerated as far as the goal state is reached after execution of the 
multi-agent plan. Alternatively, there might be domains in which other types 
of plan execution failures can occur, e.g., any change of the state not caused by 
the involved agents can be considered a failure as well. To account for the range 
of various types failures, from now on, we only require that a plan execution 
monitoring process determines some plan execution failure at a step i which 
results in some failed state s/. 

Definition 3 (multi-agent plan repair). Let II = (A,So,S g ) be a multi-agent 
planning problem. A multi-agent plan repair problem is a tuple X = (II, V, Sf,k), 
where V is a multi-agent plan solving the planning problem II, k is the step of 

V in which its execution failed and s/ € S is the corresponding failed state. 

A solution to the plan repair problem £ is a multi-agent plan V' , such that 
V' is a solution to the planning problem II' = (.4, sf,S g ). We say that V repairs 

V in Sf. In the case Plansili') = 0, we say that the plan is irreparable given the 
failure occurring at the state Sf. 

Given two multi-agent plans V\ and Vi both repairing a multi-agent plan 

V for a problem II in a state Sf, we say that V\ is preserving V more than 
V 2 iff diff{Vi,V) < iiff(J > 2,V) and denote the relation by V x < V 2 - The 
minimal repair of the multi-agent plan V is such a plan V mm € Plans(Tl'), 
which is minimal w.r.t. the mutual differences between the plans solving II'. 
I.e., Vmin G argmin diff (V ,T") . 

V'ePlans(n') 

Note, there might be several distinct minimal repairs of a given multi-agent 
plan. 
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In general, the multi-agent plan repair problem can be reduced to solving a 
modified multi-agent planning problem and thus gives rise to a straightforward 
plan repair algorithm based on replanning in two steps: 1) construct the multi- 
agent replanning problem II' as prescribed in Definition [3] and subsequently 2) 
utilize the MA-Plan algorithm (Algorithm [T] ) to solve the problem II'. 

The original motivation underlying this paper was the hypothesis that at- 
tempts to repair failed multi-agent plans lead to lower communication overhead 
than replanning. Clearly, not all planning problems could benefit from such 
a mechanism. Since we focus on multi-agent planning problems, which in a 
sense enforce coordination among the members of a multi-agent team, we firstly 
introduce the concept of fc-coordinated multi-agent planning problems. 

Definition 4 (fc-coordination). We say that a multi-agent plan V is k-coordina- 
ted iff each fragment V of length fc contains at least one joint action containing 
a public action. Formally, for every V' , s.t. V = V 'prefix ■ V 1 -V 'suffix with \V'\ = k, 
there exist i and j so that V'[i,j] G Act pub . 

We say that a multi-agent problem II is k-coordinated iff all the plans V G 
Plans(Tl) solving II, which cannot be compressed are fc-coordinated. A plan 
V G Plans(Tl) can be compressed iff it contains a fragment V , s.t. V — Vprefix • 

i ' i suffix 

and V 'prefix ■ V suffix G Plans(U). 

We informally say that multi-agent planning problems leading to plans con- 
taining coordination points (public actions) placed relatively frequently through- 
out the plans are tightly coordinated. More formally, a multi-agent planning 
problem II is tightly coordinated if it is fc-coordinated and fc is relatively low in 
comparison to the lengths of plans from Plans (II). In the case k is relatively 
high w.r.t. the plan lengths, we say that the problem is loosely coordinated and 
finally, if the plans do not involve public actions, i.e., coordination is not needed 
at all, we say the problem is uncoordinated. 

The core hypothesis of the paper can be then formulated as follows: 

Hypothesis 1. Multi-agent plan repair approaches producing more preserving 
repairs than replanning tend to generate lower communication overhead for 
tightly coordinated multi-agent problems. 

A crisper, though perhaps a more challenging version of the hypothesis would 
express the communication overhead in terms of the average communication 
complexity: 

Hypothesis 2. When applied to tightly coordinated planning problems, multi- 
agent plan repair algorithms producing more preserving repairs than replanning 
feature a lower average communication complexity than replanning. 

In the remainder of this paper, we approach resolution of Hypothesis [T] 
Treatment of Hypothesis [2] is beyond the scope of this paper and is left for 
future work. 
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Algorithm 2 Back-on-Track-Repair(E) 

Input: A multi-agent plan repair problem E = (II, "P, Sf, k), with II = 
(A, so, S g ) and a sequence of states so, . . . , s m execution of V generates in 
the ideal environment. 

Output: A multi-agent plan V' solving E if a solution exists. 

Construct Tl back = (A, s/, {s , • . . , s m }) 

if MA-Plan(n 6ac ' c ) returns a solution V back then 

Retrieve the state Sj of V to which p back returns 

return V = V back ■ V[j ...00} 
end if 



3.1 Back-on-track Repair 

Unexpected event occurring in an environment can cause a failure in execution 
of a plan performed by some multi-agent team in that environment. The result is 
that the overall state of the system is not the one expected by the undisturbed 
plan execution at the particular time step. A straightforward idea to fix the 
problem is to utilize a multi-agent planner to produce a plan from the failed 
state to the originally expected state and subsequently follow the rest of the 
original multi-agent plan from the step in which the failure occurred. The 
following multi-agent plan repair approach, coined back-on-track (BoT) repair, 
is inspired by this idea, in fact a slight generalization of it. 

Definition 5 (back-on-track repair). Let E = (Il,V,Sf,k) be a multi-agent 
plan repair problem and II' = (A, Sf,S g ) being the corresponding modified 
multi-agent replanning problem. 

We say that a plan V' S PlansQI') is a back-on-track repair of V iff there is 
a decomposition of V', such that V' = , p back ■ , P[j..oo] for some i < \V\. 

V' = r p back ■'P[i..oo] is said to be a proper back-on-track repair iff |'P[i..oo]| > 
0. I.e., V 1 preserves some non-empty suffix of V '. 

Informally, the back-on-track approach tries to preserve a suffix of the orig- 
inal plan and prefix it with a newly computed plan ~p back starting in s/ and 
leading to some state along the execution of V in the ideal environment. Note, 
that all plans from Plans(Jl') are back-on-track repairs of the original plan. The 
length of the preserved suffix of the original plan provides a handle on the re- 
pair quality ordering of the plans. The longer the preserved suffix, the more 
preserving the plan is. On the other hand, even when the plan repair problem 
E is indeed solvable, there might not be any valid proper back-on-track repair 
of the original planning problem. 

Algorithm [2] realizes a multi-agent plan repair procedure according to the 
back-on-track plan repair principle. Since the MA-Plan algorithm searches for 
the shortest plan from the initial state to a goal state, the Back-on-Track-Repair 
computes plans which return back to the original one in the shortest possible 
way. 
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Algorithm 3 Lazy-Repair(E) 

Input: A multi-agent plan repair problem £ = (II, P, Sf, k), with II = 

(A,So,S g ) and |„4| = n. 
Output: A multi-agent plan V' solving the problem £ according to the lazy 

approach, if a solution exists. 

Construct Prfe..^], the executable remainder of V from the step k and state 
s f 

Construct II ia ^ = (A, s m ,S g ) 

let V lazv be a solution to MA-Plan(II Za ^) if such exists, 
return V {k . M ■ V lazy 



3.2 Simple Lazy Repair 

The back-on-track multi-agent plan repair approach seeks to compute a new 
prefix to some suffix of the original plan and repair the failure by their con- 
catenation. An alternative approach, coined lazy, attempts to preserve the 
remainder of the original multi-agent plan and close the gap between the state 
resulting from the failed plan execution and a goal state of the original planning 
problem. 

Let Sf be the state resulting from a failure in execution of a multi-agent 
plan V in a step k. We say that a sequence of joint actions V is an executable 
remainder of V from the step k and the state Sf iff there exists a sequence of 
states Sfc, . . . , s\-p\, such that = Sf, s i+1 = Sj ®V'[i — + and for every step 
i and every agent j, we have that T"[i — k + = V[i,j] in the case P[i,j] is 
applicable in the state s, and P'[i — k + 1, j] = e otherwise. 

The following definition provides a formal definition of the lazy approach. 

Definition 6 (simple lazy repair). Let £ = (II, "P, Sf, k) be a multi-agent plan 
repair problem and II' = (A,Sf,S g ) being the corresponding modified multi- 
agent replanning problem. 

We say that a plan V' G Plans(H') is a lazy repair of V iff there is a decom- 
position of V' , such that V' = 'P[fe.. 00 i ■ 'P lazy , where 'P[fe.. 00 i is the executable 
remainder of V from the step k, execution of which results in the state si azy 
when starting in s/, and V lazy is a solution to the multi-agent planning problem 

= (A,s lazy ,s g ). 

Algorithm [3] realizes multi-agent plan repair based on the lazy repair ap- 
proach described above. 

The back-on-track approach always succeeds to compute some multi-agent 
plan repairing the original plan from the failed state in the case the replanning 
form scratch would compute such a plan from that state as well. The lazy 
approach is in this sense incomplete, as it might happen that the execution of 
the executable remainder of the original plan diverges to a state from which 
no plan to some goal state exists. Given that dynamic environment in general 
could generate irreparable failures, this incompleteness cannot be considered a 
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shortcoming of the lazy approach in general. Of course in domains in which no 
irreparable unexpected event might occur, while at the same time the agents 
are allowed to perform actions potentially having irreversible and potentially 
harmful effects, the lazy approach has to be employed with caution. 

3.3 Repeated Lazy Repair 

In dynamic environment plan failures occur repeatedly, i.e., even after a repair 
of a failed plan, it is possible for the repaired plan to fail again. In this situation 
both the back-on-track, as well as the lazy multi-agent plan repair algorithms 
lead to prolonging the really executed plan. In the case of the back-on-track 
approach, this is inevitable, since upon the repair, the subsequent plan execution 
process immediately processes the newly added plan fragment. In the case of 
the lazy repair, however, upon occurrence of another failure during execution of 
the repaired plan, it is not always necessary to prolong the overall multi-agent 
plan. 

The intuition behind the repeated lazy plan repair approach is that a failure 
during execution of an already repaired plan makes the previous repair attempt 
irrelevant and its result can be discarded, unless the failure occurred already 
in the plan fragment appended by the previous repair. The following definition 
formally introduces the extension of the lazy multi-agent plan repair approach. 
For clarity, from now on, we refer to the lazy multi-agent plan repair approach 
introduced in Definition [6] as simple lazy repair. 

Definition 7 (Repeated lazy repair). Let II = (A,so,S g ) be a multi-agent 
planning problem with a solution V and Ej = (II, V, S/ x , fci) be a multi-agent 
plan repair problem with a lazy repair solution Vs ± = "Pnfe 1 .. 0o ] ■'Psuffix and £2 = 
(II, Ps-l, Sf 2 , ^2) be a multi-agent plan repair problem the system is currently 
facing. 

We say that V' is a repeated lazy repair of V\ iff 

1. V' is a simple lazy repair solution to £' = (II, V, s/ 2 , fo) in the case ki < 

l^fcL.oo]!; and 

2. V' is a simple lazy repair solution to S 2 otherwise. 

The repeated lazy repair leads to a straightforward extension of the lazy 
plan repair algorithm listed in Algorithm [3] Note, that the repeated lazy repair 
algorithms enables a plan execution model which preserves significantly longer 
fragments of the original plan. That is, upon a failure, instead of trying to repair 
the failed plan right away, as both the back-on-track and simple lazy plan repair 
algorithms do, the system can simply proceed with execution of the remainder 
of the original plan and only after its complete execution the lazy plan repair is 
triggered. The approach simply ignores the plan failures during execution and 
postpones the repair the very end of the process, hence the "lazy" label for the 
two algorithms. 
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Algorithm 4 Repeated-Lazy-Repair(Ei, , S2) 

Input: A multi-agent planning problem II = (A,so,S g ), a multi-agent plan 
V = V[k..oc] ' suffix solving II, and two multi-agent plan repair problems 
Ei = (II, P, s/i, ki) with a solution Vs 1 and £ 2 = (n, , s/ 2 , fc 2 ). 

Output: A multi-agent plan solving £ 2 if a solution exists. 

if fc 2 < \V[k..oo]\ then 

Construct £' = (II, V, s/ 2 , k%) 
else 

£' = S 2 
end if 

return a solution to Lazy-Repair(E') if such exists 



4 Experimental validation 

To verify the Hypothesis [T] we conducted a series of experiments with imple- 
mentations of the multi-agent plan repair algorithms described in Section [3] 
Below, we firstly describe the experimental setup used for the experiments and 
subsequently interpret the data collected and revisit Hypothesis [TJ 

4.1 Experimental Setup 

The experiments were based on a two-stage algorithm. In the first phase, for 
a given domain a multi- agent plan was computed using the MA-Plan algorithm 
based on a distributed constraint satisfaction solver for computing the candidate 
coordination plans and implementation of a best-first-search action planning 
algorithm as part of FF [6] for computing the local, single-agent plans. We 
used the implementation of the distributed multi-agent planner authored by 
Nissim et al. also used for the experiments conducted in their paper [TT]. In 
the second phase, we executed the multi-agent plan. In the course of the plan 
execution, we simulated the environment dynamics by producing various plan 
failures according to a variable failure probability. The plan execution was 
monitored and upon a failure detection a plan repair algorithm was invoked. 
Algorithm [5] lists the pseudo-code of the process. 

Before execution of each plan step, the joint action is checked for applicability 
in the current state. In the case it is not applicable, a plan repair algorithm is 
invoked and the execution continues on the repaired plan. Otherwise, the state 
is updated with the joint action. 

The execute-fail function in the algorithm either updates the current state 
by the joint action provided as a parameter as if in the ideal environment, or to 
generate an unexpected event occurring in the simulated dynamic environment. 

We distinguish two types of plan failures: action failures and state perturba- 
tions. Both failure types are parametrized by a uniformly distributed probability 
P, which determines whether a simulation step fails, or not. Both failure types 
are weak failures. That is, they are not handled immediately, but can preclude 
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Algorithm 5 Plan execution and monitoring algorithm. 
Input: An initial multi-agent planning problem II = (.4, So, S g ). 

V = MA-Plan(n) 

s = so! s t e P = 1 
repeat 

if <f>v[ste P ] % s then 

V = Repa\r(U,V,s,step) 
step = 1 
end if 

s = execute-fail(s, V[step\) 
step — step + 1 
until step > \V\ 



the plan execution and later result in a strong failure. Upon detection, a strong 
failure is handled by one of the plan repairing algorithms. 

An action failure is simulated by not-executing some of the individual agent 
actions from the actual plan step. The individual action is chosen according to 
a uniformly distributed probability. The individual action is removed from the 
joint action and the current state is updated by the modified joint action. 

The other simulated failure type, state perturbation, is parametrized by a 
positive non-zero integer c, which determines the number of state terms, which 
are removed from the current state, as well as the number of terms which are 
added to it. The terms to be added or removed are selected also randomly from 
the domain language according to a uniform distribution. 

The experimental setup was implemented as a centralized simulator of the 
environment integrating a decentralized multi-agent domain-independent plan- 
ner MA- Plan. The individual agents are initialized by a planning domain, to- 
gether with a particular planning problem instance. Each agent runs in its 
own thread and they deliberate asynchronously. The agents send peer-to-peer 
messages among themselves. Message passing is mediated by the centralized 
simulator as well. The messages are sent in the DisCSP phase by the integrated 
solver, which is a part of the MA- Plan planner. 

The experiments were performed on Phenom Quad Core 9950 processor at 
2.6GHz with Java Virtual Machine limited to 2.5GB of RAM. The individual 
measurements were parametrized by the plan failure probability P and each 
problem instance was executed 6-10 with various value samples. The resulting 
data are, in the figures, presented with natural distribution. The candlestick 
charts depict the difference between the minimal and the maximal measure- 
ments, together with the standard deviation. 

4.2 Test Problems, Algorithms and Metrics 

The experiments were conducted on three planning domains. The domains 
originate in the standard benchmark single-agent ICP planning domains pub- 
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lished at pQ. Similarly to [TT], we chose domains, which are straightforwardly 
modifiable to multi-agent planning problems: LOGISTICS (3 agents), ROVERS (3 
agents), and satellites (2-5 agents). 

The LOGISTICS domain is a tightly coordinated in that it requires relatively 
frequent coordination among the involved agents: airplanes and trucks need to 
wait for each other to load or unload the transported packages. The ROVERS 
domain is loosely coordinated in that it requires coordination only at the end of 
plans: there is a single shared communication channel between one of the rovers 
and the receiving station. Finally, the satellites domain is uncoordinated in 
that it does not need any coordination between the satellites acquiring images 
individually. 

To evaluate validity of Hypothesis [T] the multi-agent planning problems were 
tested on the experimental setup against a plan repair algorithm implementing 
rcplanning from scratch and two of the repair algorithms Back-on-Track-Repair 
(Algorithm [2]) and Repeated-Lazy-Repair (Algorithm [4]) introduced in Section[3] 

Efficiency problems of the MA-Plan implementation limited the experiments 
to plans with maximally two landmarks (coordination points). The measure- 
ments of Back-on-Track-Repair algorithm runs were negatively influenced by 
sensitivity of the planner implementation to the number of terms in the goal 
state. Additionally, the Back-on-Track-Repair algorithm could not leverage dis- 
junctive goal form (cf. Definition |5| and this was emulated by iterative process 
testing all term conjunctions in a sequence and thus resulting in multiple runs 
of the DisCSP solver instead of a single run with disjunctive goal. 

We used three metrics to evaluate the measurements: 

execution length is the overall number of joint actions the experimental setup 
executed. 

planning time is the measured cumulative time consumed by the underlying 
MA-Plan planner used for generating initial and repairing plans; and finally 

communication is measuring the number of messages passed between the 
agents during the planning or plan repair process. That is messages gen- 
erated by the DisCSP solver in the MA-Plan planner. 

4.3 Results and Discussion 

The first batch of experiments directly targets validation of Hypothesis [T] multi- 
agent plan repair is expected to generate lower communication overhead in tightly 
coordinated domains. We used LOGISTICS as a tightly coordinated domain and 
dynamics of the simulated environment modeled as action failures. Figure [T] 
depicts the results of the experiment. The communication overhead generated 
by the Back-on-Track-Repair algorithm is on average only 59% (36% at best) of 
that generated by the replanning approach. Furthermore, the Repeated- Lazy- 
Repair algorithm performed even better and on average produced only 43% (11% 
at best) of the communication overhead generated by the replanning algorithm. 
In result, the experiments strongly support our hypothesis. 
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Figure 1: Experimental results for LOGISTICS domain with 3 agents and action 
failures. 
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Figure 2: Experimental results for ROVERS domain with 3 agents and action 
failures (top). Experimental results for LOGISTICS domain with 3 agents and 
state perturbations with c = 1 (bottom) . 
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Figure 3: Experimental results for LOGISTICS domain with 3 agents and state 
perturbations with c = 1. 



Additionally, the overall time spent in the planning phase (used by the MA- 
Plan algorithm) by the plan repair algorithms was 54% (34% at best) and 51% 
(12% at best) for Back-on-Track-Repair and Repeated-Lazy-Repair respectively. 
The execution length was lower in comparison the replanning approach as well 
being in average 96% (72% at best, 130% at worst) by Back-on-Track-Repair and 
lower being 81% (34% at best, 132% at worst) for Repeated-Lazy-Repair. 

The second batch of experiments focused on boundaries of validity of the 
positive result presented above. In particular, we validated the condition on 
the coordination tightness and feasibility of failures. The auxiliary hypothesis 
we validated states: with decreasing coordination tightness of the domain, the 
communication efficiency gains of repairing techniques should decrease. For 
loosely coordinated domains the communication efficiency of plan repair should 
be on-par with that of the replanning approach. 

To validate the auxiliary hypothesis we ran experiments with the rovers 
as a loosely coordinated domain. Figure [3] (top) presents the results supporting 
the hypothesis. 

The third batch of experiments targeted the perturbation magnitude of the 
plan failures. The second auxiliary hypothesis we validated states: communica- 
tion efficiency gain of plan repairing in contrast to replanning should decrease 
as the difference between a nominal and related failed state increases. The un- 
derlying intuition is that, in the case the dynamic environment generates only 
relatively small state perturbations and the failed states are "not far" from the 
actual state, the plan repair should perform relatively well. On the other hand, 
if the state essentially "teleports" the agents to completely different states, re- 
planning tends to generate more efficient solutions than plan repair. 

To answer this hypothesis, we have prepared another LOGISTICS experiment 
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employing state perturbations as the model of the environment dynamics. Fig- 
ure [3] (bottom) depicts results of the experiment for c = 1 . The perturbed state 
for c = 1 is produced by removing one term from the actual state and adding 
another one. As the chart shows, under random perturbations the plan repairing 
technique lost its improvement against replanning. For stronger perturbations 
with c = 2,3, 4, the ratio between plan repairing and replanning remained on 
average the same. The trend of the absolute numbers of messages, planning time 
and execution length was slightly decreasing, as the probability of opportunistic 
effects increased. 

Finally, we conducted a series of experiments with a non-coordinated SAT- 
telites domain. The results depicted in Figure [3] show the anticipated lower 
plan repair communication efficiency in contrast to replanning. 

5 Final remarks 

In the presented paper, we formally introduced the problem of multi-agent plan 
repair, proposed three algorithms for solving it and experimentally validated 
the hypothesis stating that under certain conditions, multi-agent plan repair 
approach tends to be more efficient in terms of the communication overhead 
it generates in comparison to the replanning approach. Our results well sup- 
port the core hypothesis of the paper and we additionally performed a series of 
experiments validating its boundary conditions. 

The line of research underlying this paper well correlates with recent works 
on classical single-agent planning sub-domains, such as partial ordered plan 
monitoring and repairing [9 , conformant and contingency planning, plan re- 
use and plan adaptation. Environment dynamics is also handled by approaches 
based on Markov decision processes. The main difference to our approach is 
that the state perturbations utilized in our experiments have a priori unknown 
probabilities. Our own recent approach to the problem of multi-agent plan repair 
in [7] can be seen only as a precursor to the formal and rigorous treatment of 
the problem in this paper. Therein, we described the first steps towards formal 
treatment of the problem, as well as proposed two specific incomplete algorithms 
for solving the problem, very distinct from the ones presented here. 

There are several open challenges resulting from the presented work. Firstly, 
the multi-agent planning framework (MA-Strips) is not expressive enough to 
describe certain aspects of concurrent actions and should be extended to this 
end, what, we suspect, will also influence the multi-agent planning complexity 
analysis. In particular, there is no way to account for joint actions which have 
effects strictly different than the unity of the individual actions involved. An- 
other issue is that there is no way to enforce or forbid concurrent execution of 
certain individual actions. Secondly, the framework is not able to describe con- 
current resource consumption, which is not an issue in single-agent Strips [3] 
planning, but in the multi-agent extension two individual concurrently executed 
actions might "consume" the same precondition, even though it is undesirable 
in the domain. Thirdly, there is a need for more efficient implementations of 
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multi-agent planners with more features as the gap between the state-of-the- 
art classical planners and multi-agent planners is enormous. Fourthly, there 
is a lack of standardized planning benchmarks for multi-agent planning, espe- 
cially considering tightly coordinated planning problems. Such are needed to 
further evaluate the hypotheses presented in this paper. Finally, we leave out 
the work towards resolving the validity of Hypothesis [2] aiming at investigation 
of complexity issues of multi-agent plan repair to future work. 
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